Lensless imaging device for microscopy and fingerprint biometric

ABSTRACT

In one aspect, embodiments disclosed herein relate to a lens-free imaging system. The lens-free imaging system includes: an image sampler, a radiation source, a mask disposed between the image sampler and a scene, and an image sampler processor. The image sampler processor obtains signals from the image sampler that is exposed, through the mask, to radiation scattered by the scene which is illuminated by the radiation source. The image sampler processor then estimates an image of the scene based on the signals from the image sampler, processed using a transfer function that relates the signals and the scene.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/368,618 filed Jul. 29, 2017, the entire disclosure of which ishereby expressly incorporated by reference herein.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

The invention was made with government support under Grant NumberCCF-1527501 awarded by the National Science Foundation, N00014-15-1-2878awarded by the Department of Defense, and EAGER awarded by the NationalScience Foundation. The government has certain rights in the invention.

BACKGROUND

Imaging devices such as microscopes are used in various disciplines,such as biology. However, traditional microscopes suffer from afundamental tradeoff between size and performance. More specifically,down-sizing conventional microscopes typically results in the collectionof less light and/or the imaging of a smaller field of view. Overcomingthis tradeoff would be desirable and would make microscopy suitable fornovel applications.

SUMMARY

This summary is provided to introduce a selection of concepts that arefurther described below in the detailed description. This summary is notintended to identify key or essential features of the claimed subjectmatter, nor is it intended to be used as an aid in limiting the scope ofthe claimed subject matter.

In one aspect, embodiments disclosed herein relate to a lens-freeimaging system. The lens-free imaging system includes an image sampler;a radiation source; a mask disposed between the image sampler and ascene; and an image sampler processor that: obtains signals from theimage sampler that is exposed, through the mask, to radiation scatteredby the scene, wherein the scene is illuminated by the radiation source;and estimates an image of the scene based on the signals from the imagesampler, processed using a transfer function that relates the signalsand the scene.

In another aspect, embodiments of the present disclosure relate tomethods for generating an image of a scene using a lens-free imagingsystem. The method includes: obtaining signals from an image samplerthat is exposed, through a mask, to radiation scattered by the scene;and estimating the image of the scene, based on the signals from theimage sampler, processed using a transfer function that relates thesignals and the scene.

BRIEF DESCRIPTION OF DRAWINGS

Certain embodiments of the invention will be described with reference tothe accompanying drawings. However, the accompanying drawings illustrateonly certain aspects or implementations of the invention by way ofexample and are not meant to limit the scope of the claims.

FIG. 1 shows a diagram of a lens-less imaging system in accordance withone or more embodiments of the invention.

FIG. 2 shows a diagram of a second lens-less imaging system inaccordance with one or more embodiments of the invention.

FIG. 3 shows a diagram of the system of FIG. 1 in accordance with one ormore embodiments of the invention.

FIG. 4 shows a diagram of an exemplary image sampler in accordance withone or more embodiments of the invention.

FIG. 5 shows a diagram of an exemplary mask in accordance with one ormore embodiments of the invention.

FIG. 6 shows a diagram of an image sampler processor in accordance withone or more embodiments of the invention.

FIGS. 7A-7C shows calibration setups in accordance with embodiments ofthe invention.

FIG. 8 shows a flowchart of a method for calibrating a lens-free imagingsystem in accordance with embodiments of the invention.

FIG. 9 shows a flowchart of a method for generating an image using alens-free imaging system in accordance with embodiments of theinvention.

DETAILED DESCRIPTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures. Like elements in the variousfigures are denoted by like reference numerals for consistency. Likeelements may not be labeled in all figures for the sake of simplicity.

In the following detailed description of embodiments of the invention,numerous specific details are set forth in order to provide a morethorough understanding of the invention. However, it will be apparent toone of ordinary skill in the art that the invention may be practicedwithout these specific details. In other instances, well-known featureshave not been described in detail to avoid unnecessarily complicatingthe description.

Throughout the application, ordinal numbers (e.g., first, second, third,etc.) may be used as an adjective for an element (i.e., any noun in theapplication). The use of ordinal numbers is not to imply or create anyparticular ordering of the elements nor to limit any element to beingonly a single element unless expressly disclosed, such as by the use ofthe terms “before”, “after”, “single”, and other such terminology.Rather, the use of ordinal numbers is to distinguish between theelements. By way of an example, a first element is distinct from asecond element, and the first element may encompass more than oneelement and succeed (or precede) the second element in an ordering ofelements.

In the following description of FIGS. 1-9, any component described withregard to a figure, in various embodiments of the technology, may beequivalent to one or more like-named components described with regard toany other figure. For brevity, descriptions of these components will notbe repeated with regard to each figure. Thus, each and every embodimentof the components of each figure is incorporated by reference andassumed to be optionally present within every other figure having one ormore like-named components. Additionally, in accordance with variousembodiments of the technology, any description of the components of afigure is to be interpreted as an optional embodiment which may beimplemented in addition to, in conjunction with, or in place of theembodiments described with regard to a corresponding like-namedcomponent in any other figure.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a horizontal beam” includes referenceto one or more of such beams.

Terms such as “approximately,” “substantially,” etc., mean that therecited characteristic, parameter, or value need not be achievedexactly, but that deviations or variations, including for example,tolerances, measurement error, measurement accuracy limitations andother factors known to those of skill in the art, may occur in amountsthat do not preclude the effect the characteristic was intended toprovide.

It is to be understood that, one or more of the steps shown in theflowcharts may be omitted, repeated, and/or performed in a differentorder than the order shown. Accordingly, the scope of the inventionshould not be considered limited to the specific arrangement of stepsshown in the flowcharts.

Although multiple dependent claims are not introduced, it would beapparent to one of ordinary skill that the subject matter of thedependent claims of one or more embodiments may be combined with otherdependent claims.

Embodiments of the invention relate to lens-less imaging systems,lens-less imaging sensors, and methods of operating lens-less imagingsystems. A lens-less imaging system may generate electrical signals inresponse to receiving electromagnetic (EM) radiation. The electricalsignals generated by the lens-less imaging system may be processed togenerate an image of a scene from which the electromagnetic energy wasscattered.

In one or more embodiments of the invention, a lens-less imaging systemmay include an image sampler, one or more masks, a radiation source, andan image sampler processor. Each of the aforementioned components may beused to generate an image of a scene. The image sampler may generateelectronic signals in response to receiving EM radiation. The imagesampler may be, for example, a charge capture device (CCD) that respondsto EM radiation. In one or more embodiments of the invention, the imagesampler may be a sensor. In one or more embodiments of the invention,the sensor may be a sensor array. Other devices may be used as the imagesampler without departing from the invention.

In one or more embodiments of the invention, the EM radiation may bevisible light, short-wave infrared light, mid-wave infrared light,long-wave infrared light, terahertz, millimeter wave, wireless or otherparts of the electromagnetic spectrum. The EM radiation may be generatedby a radiation source. The radiation source may be, for example, one ormore micro light emitting diodes. The radiation source may be integratedinto the image sampler, may be disposed adjacent to the image sampler,or may be disposed so that radiation is transmitted through a scene andreceived by the image sampler. The radiation source may be a lightemitting source. In one embodiment of the invention, the radiationsource may be the scene itself, e.g. in case of a bioluminescent samplebeing examined.

In one or more embodiments of the invention, each of the masks may be aplanar structure disposed between the image sampler and a scene. In oneor more embodiments of the invention, the masks may include regions thatare opaque and other regions that are transparent to EM radiation. Theopaque and transparent regions may cause EM radiation that is incidenton the mask to be partially reflected away from the image sampler andpartially transmitted to the image sampler.

In one or more embodiments of the invention, the masks may includeregions that have different refractive indices or thickness oftransparent materials. The different regions having different refractiveindices or thickness may generate amplitude and phase modulation ofincident EM radiation. EM radiation that is transmitted to the imagesampler may be diffracted and/or amplitude/phase modulated when comparedto EM radiation that is incident on the masks.

In one or more embodiments of the invention, the mask may be disposed ata predetermined distance from the image sampler. The predetermineddistance may be, for example, 500 nm. In one or more embodiments of theinvention, the predetermined distance is less than 1 μm. In one or moreembodiments of the invention, the predetermined distance is less than 5μm. In one or more embodiments of the invention, the predetermineddistance is less than 500 μm. In one or more embodiments of theinvention, a spacer may be disposed between the mask and the imagesampler. The spacer may be transparent or translucent to EM radiation towhich the image sampler is responsive. The invention is not limited tothe aforementioned examples.

In one or more embodiments of the invention, the image sampler processormay generate an image of the scene based on the electronic signalsreceived from the image sampler. In one or more embodiments of theinvention, the image sampler processor may be a sensor processor. Theimage sampler processor may be, for example, a digital signal processor(DSP), a general purpose processor, an application integrated circuit, afield-programmable gate array, an analog circuit, or any other type ofelectronic circuit. In one or more embodiments, the electronic signalsmay be received directly from the image sampler. In one or moreembodiments, the electronic signals may be stored on a storage andretrieved from the storage by the image sampler processor. The imagesampler and image sampler processor may be disposed at differentlocations without departing from the invention.

In one or more embodiments, the scene may be disposed adjacent to themask, or in close proximity to the mask, e.g. at a distance less than 5mm from the mask. For example, a specimen, sample, or other object ofinterest may be placed in close proximity to the mask. In one or moreembodiments, the scene may be disposed directly adjacent to the mask.For example, a specimen, sample, or other object of interest may beplaced in direct contact with the mask. In one or more embodiments ofthe invention, the scene may be a biometric identifier. The biometricidentifier may be a finger print. In one or more embodiments of theinvention, the scene may be tissue such as muscle or nerve tissue. Thenerve tissue may be, for example, brain tissue. The tissue may be invivo. The scene may be other tissues without departing from theinvention.

Additional embodiments of the invention may relate to a method ofoperating a lens-less imaging system. The method may include displayinga number of known scenes to the lens-less imaging system and obtainingthe electrical signal produced by an image sampler while exposed to eachof the known scenes. The transfer function that relates EM radiationthat is radiated by an unknown scene and electrical signals produced byan image sampler of the lens-less imaging system may be determined usingthe obtained electrical signals.

FIG. 1 shows a lens-less imaging system in accordance with one or moreembodiments of the invention. The system may be configured to generatean image of a scene (100). The system may include an image sampler(140), one or more masks (120), a spacer (130), and an image samplerprocessor (150). Each of the aforementioned components of the system aredescribed below.

The image sampler (140) may be a physical device that generateselectrical signals in response to incident EM radiation. In one or moreembodiments of the invention, the image sampler (140) includes a numberof sensing regions (142) and a number of EM radiation generation regions(141). The image sampler (140) may be operatively connected to the imagesampler processor (150). The image sampler processor (150) may controlthe operation of the image sampler (140).

The EM radiation generation regions (141) may be, for example, lightemitting diodes or organic light emitting diodes. The EM radiationgeneration regions (141) may be other EM radiation generating structureswithout departing from the invention. Portions of the EM radiationgeneration regions (141) may be configured to generate a first type ofEM radiation, e.g., green light, while other portions maybe configuredto generation a second type of EM radiation, e.g., red light. The EMradiation generation regions (141) may be divided into any number ofregions that each generate different types of radiation withoutdeparting from the invention.

The sensing regions (142) may each be, for example, a charge-coupleddevice (CCD) or a CMOS device that generates an electrical signal whenexposed to EM radiation. The sensing regions (142) may generateelectrical signals that represent an amplitude, phase, and/or frequencyof received EM radiation. In one or more embodiments of the invention,the sensing regions (142) may be divided into portions. Each of theportions may be tuned to have a high degree of sensitivity to apredetermined frequency of EM radiation.

FIG. 4 shows an example of an image sampler in accordance with one ormore embodiments of the invention. The image sampler (140) includes anumber of sensing regions (142) that each generate electrical signals inresponse to EM radiation incident on each region. For example, EMradiation may be incident on a first EM sensing region and a second EMsensing region. The first EM sensing region may produce a firstelectrical signal and the second EM sensing region may produce a secondelectrical signal. Each of the signals may be proportional to amagnitude and/or frequency of the EM radiation that is incident on eachEM sensing region.

The sensing regions (142) of the image sampler (140) may be disposedover a two-dimensional (2D) area. While the sensing regions (142) areshown as square regions in a uniform 2D grid, the sensing regions (142)may have other shapes and may be distributed in a uniform or irregularpattern over the surface without departing from the invention. In one ormore embodiments of the invention, the 2D area may be a flat surface. Inone or more embodiments of the invention, the 2D area may be a curvedsurface.

The image sampler (140) also includes a number of EM radiationgeneration regions (141) that each generate EM radiation. The EMradiation may illuminate a scene disposed on the masks (120). Theoperation of the EM radiation generation regions (141) may be controlledby the image sampler processor (150) by sending instructions to theimage sampler (140) via the operable connection.

In one or more embodiments of the invention, the sensing regions (142)of the image sampler (140) may generate electrical signals in responseto visible light. In one or more embodiments of the invention, thesensing regions (142) of the image sampler (140) may generate electricalsignals in response to infrared radiation. In one or more embodiments ofthe invention, the sensing regions (142) of the image sampler (140) maygenerate electrical signals in response to ultraviolet light, short-waveinfrared light, mid-wave infrared light, long-wave infrared light,wireless, millimeter wave or terahertz radiation. The sensing regions(142) of the image sampler (140) may generate electrical signals inresponse to EM radiation having other spectral content without departingfrom the invention. Those skilled in the art will appreciate that theimage sampler (140) may be sensitive to any type of radiation, includingelectromagnetic and particle radiation, ionizing and non-ionizingradiation, and combinations thereof. The imagine sampler may, forexample, be sensitive to multiple wavelengths to produce a color image.

In one or more embodiments of the invention, at least one EM radiationgeneration region of the EM radiation generation regions (141) may beinterposed between the sensing regions (142). In one or more embodimentsof the invention, at least one EM radiation generation region of the EMradiation generation regions (141) may be disposed adjacent to at leastone sensing region of the sensing regions (142). In one or moreembodiments of the invention, the EM radiation generation regions (141)may be spatially dispersed to illuminate a scene. For example, each ofthe EM radiation generation regions (141) may be disposed in a gridpattern. In another example, each of the EM radiation generation regions(141) may be disposed in an outline pattern. The outline pattern may be,for example, the outline of a box, circle, rectangle, or any othershape. The EM radiation generation regions (141) may be disposed inother patterns to illuminate the scene without departing from theinvention.

Returning to FIG. 1, the system may include one or more masks (120). Themasks (120) may be a physical device that masks portions of the imagesampler (140) from EM radiation that is incident on the masks (120). Themasks (120) may be disposed between the scene (100) and the imagesampler (140) and thereby mask portions of the image sampler (140) fromEM radiation from the scene (100). A spacer (130) may be disposedbetween the masks (120) and the image sampler (140). In one or moreembodiments of the invention, the thickness of the spacer is less than500 nm. In one or more embodiments of the invention, the thickness ofthe spacer is less than 1000 nm. In one or more embodiments of theinvention, the thickness of the spacer is less than 5000 nm. In one ormore embodiments of the invention, the thickness of the spacer is lessthan 500 μm. The invention is not limited to the aforementioneddistances. A glass plate may further be disposed between the mask (120)and the sample (100). The glass plate may form a cover of the lens-freeimaging system and the sample may be placed over the glass plate. Theglass plate may further be a slide, e.g., a slide that contains abiological sample such as tissue, blood, etc. The glass plate mayfurther be or form a channel for flow cytometry, enabling a fluid usedto transport cells or other material across the lens-free imagingsystem.

FIG. 5 shows a diagram of an example of a mask of the masks (120) inaccordance with one or more embodiments of the invention. In one or moreembodiments of the invention, the mask is a two-dimensional pattern ofrefractive index changes that change the effective path length ofradiation through the mask of radiation through the mask. In one or moreembodiments of the invention, the mask is a two-dimensional pattern ofamplitude-changing elements that change the effective transparency toradiation through the mask. In one or more embodiments of the invention,the mask is a two-dimensional pattern of phase-changing that change theeffective phase of radiation through the mask. In one or moreembodiments of the invention, the mask is a two-dimensional pattern ofscatterers that change the direction of radiation through the mask. Inone or more embodiments of the invention, the mask is a threedimensional volumetric pattern. In one or more embodiments of theinvention, the mask is dynamic and changes over time.

In one or more embodiments of the invention, the mask may include firstregions (500) that are opaque and second regions (510) that aretransparent. The opaque first regions (500) may prevent the transmissionof EM radiation through the mask and the transparent second regions(510) may allow the transmission of EM radiation through the mask.

In one or more embodiments of the invention, opaque first regions (500)may be gold, aluminum, chrome, or other metals. In one or moreembodiments of the invention, the transparent second regions (510) maybe glass, quartz, or may be a void space.

In one or more embodiments of the invention, the mask includes firstregions (500) that have refractive indices or thicknesses that aredifferent than the refractive indices and/or thicknesses of secondregions (510) that are transparent. The first regions (500) and thesecond regions (510) may amplitude and/or phase module EM radiation thatis transmitted through the mask.

In one or more embodiments of the invention, the first regions (500) andthe second regions (510) may be distributed over a two dimensionalsurface. Each region may be disposed adjacent to other regions. Whilethe first regions (500) and second regions (510) are shown in FIG. 5 ina specific two dimensional pattern, the first regions (500) and secondregions (510) may be distributed in other two dimensional patterns orrandomly distributed without departing from the invention.

In one or more embodiments of the invention, the first regions (500) andsecond regions (510) are distributed across the mask specified by thepattern Min the following equation:

$\begin{matrix}{{M = \frac{1 + {m_{1}m_{2}^{T}}}{2}},} & (1)\end{matrix}$

where matrix M is of dimensions N×N. In one embodiment of the invention,m₁ and m₂ are each 1D sequences where each element is either +1(transparent, member of the first regions (500)) or −1 (opaque, memberof second regions (510)). Because opaque apertures block light, they maybe assigned the value “0”, in equation 1.

A separable mask pattern, composed of the cross-product of two 1Dfunctions simplifies the calibration and image reconstruction process,as further described below, in accordance with one or more embodimentsof the invention.

In one or more embodiments of the invention, 50% of the elements of themask pattern M specify 0, e.g., an opaque region, and 50% of theelements of the mask specify 1, e.g., a transparent region.

In one or more embodiments m₁ and m₂ are random sequences where eachelement of the sequence specifies +1 or −1. In one or more embodimentsm₁ and m₂ are maximum length sequences. In one or more embodiments m₁and m₂ establish a modified uniform redundant array (MURA).

In one or more embodiments of the invention, there may be an equalnumber of first regions (500) and second regions (510). In other words,if there are 8000 first regions (500) there are 8000 second regions(510).

In one or more embodiments of the invention, each opaque region (500)and each transparent region (510) may have a square cross section. Thesquare cross section of each region may have an edge length of thefeature size of each region. In other words, a feature size of 30 nmindicates that each region is a 30 nm×30 nm square.

In one or more embodiments of the invention, the masks (120) may bedisposed on a transparent support structure. The transparent supportstructure may be, for example, a quartz wafer.

Returning to FIG. 1, the system may include an image sampler processor(150). The image sampler processor (150) may be a physical device. Theimage sampler may estimate a scene based on electrical signals receivedfrom the image sampler (140) and control the operation of the imagesampler (140). Specifically, the image sampler processor (150) mayactivate/deactivate the EM radiation generation regions (141) of theimage sampler (140).

FIG. 6 shows a diagram of an image sampler processor (150) in accordancewith one or more embodiments of the invention. The image samplerprocessor (150) may be configured to and/or programmed to generateestimates e.g., images, of a scene (100) based on electrical signalsreceived from the image sampler (140). The image sampler processor (150)may include a processor (600), transfer function (610), and storage(620). Each of the aforementioned components of the system are describedbelow.

In one or more embodiments of the invention, the processor (600) may bea general purpose processor, embedded processor, programmable gatearray, digital signal processor, or any other type of data processingdevice.

The storage (620) may be a non-transitory computer readable storageincluding instruction that, when executed by the processor (600), causethe image sampler processor (150) to perform the functionality shown inFIGS. 8 and 9, and described below. While the storage (620) isillustrated as being part of the image sampler processor (150), thestorage (620) may be remote without departing from the invention.

When executing the instructions stored on the storage (620), the imagesampler processor (150) may generate an estimate e.g., an image, of thescene (100) based on a relationship between EM radiation scattered bythe scene (100) and the electrical signals generated by the imagesampler (140). When EM radiation interacts with the scene (100), it maybe scattered. A portion of the scattered EM radiation may be scatteredtoward the system as scene EM radiation.

When scene EM radiation is incident on the system, the masks (120) mayscatter some of the scene EM radiation away from the image sampler (140)as reflected EM radiation and may scatter some of the scene EM radiationtowards the image sampler (140) as transmitted radiation. The reflectedEM radiation does not interact with the image sampler (140) while thetransmitted EM radiation interacts with the image sampler (140), e.g.,causes the image sampler (140) to generate electrical signals. Thetransmitted EM radiation may be modulated and diffracted by the masks(120) when compared to the scene EM radiation. Modulating anddiffracting the transmitted EM radiation may multiplex the transmittedradiation onto a number of sensing regions of the image sampler.

Additionally, when the scene is disposed directly on the masks (120),each sensing region of the scene only receives scene radiation from aportion of the scene. For example, as shown in FIG. 3, the image sampler(140) includes a number of sensing regions (300). An example sensingregion (300) only receives EM radiation from a portion of the scene(320) due to the proximity of the scene. When the scene (100) isdisposed close to the image sensor, a portion of the EM radiationscattered by the scene (100) is incident on the masks at an obliqueangle that causes the portion of the EM radiation to scatter away fromthe image sampler (140) rather than be transmitted toward the imagesampler (140). In contrast, when a scene is disposed away from themasks, all of the EM radiation scattered by the scene is normallyincident on the mask which enables transmission of the scattered EMradiation to the image sampler (140).

To generate an image of the scene (100), it may be assumed that thescene includes a number of EM radiation sources equal to the number offeatures of the masks (120). In other words, if the mask is a 100×100feature array, it may be assumed that the scene consists of 100×100radiation sources.

Based on the above and further assuming that a separable mask pattern,composed of the cross-product of two 1D functions, as previouslydescribed, is used, the local spatially varying point spread functioncan be decomposed into two independent, separable terms. The first termmodels the effect of a hypothetical “open” mask (no apertures), and thesecond term models the effect due to the coding of the mask pattern. Fora 2D (i.e., planar sample X_(d), at depth d, the transfer functionbetween the planar sample (i.e., the scene (100)) and the electricalsignals generated by the image sampler (140) is:

Y=P _(od) X _(d) Q _(od) ^(T) +P _(cd) X _(d) Q _(cd) ^(T),  (2)

in accordance with an embodiment of the invention. P_(od) and P_(cd) mayoperate only on the rows of X_(d), and Q_(od) and Q_(cd) may operateonly on the columns of X_(d). The subscripts o and c refer to “open” and“coding,” respectively. Importantly, the total number of parameters inP_(od), Q_(od), P_(cd) and Q_(cd) is O(N²) instead of O(N⁴). Thus,calibration of a moderate resolution lens-less imaging system with a 1megapixel sensor requires the estimation of only ˜4×10⁶ rather than 10¹²elements, and image reconstruction requires roughly 10⁹ instead of 10¹⁸computations.

In one embodiment of the invention, equation 2, which is specific to aplanar 2D sample X_(d) at depth d, is generalized to 3D space. For a 3Dvolumetric sample X_(D), a series of planar samples X_(d) for Ddifferent depths may be superpositioned to obtain measurements for the3D volume. Equation 2, for the 3D generalization, thus, becomes

Y=Σ _(d=1) ^(D)(P _(od) X _(d) Q _(od) ^(T) +P _(cd) X _(d) Q _(cd)^(T)).  (3)

The image of the scene may be determined by inverting equations 2 or 3,assuming the calibration matrices are known. For example, the image ofthe scene may be determined by recording the electrical signals of theimage sampler (140) when the system is exposed to scene EM radiation.Equations 2 and 3 may be inverted using any method such as aleast-squares technique or other regularized least-squares techniques.An appropriate regularization may be chosen based on the scene. Forexample, for extended scenes such as the USAF resolution target, aTikhonov regularization may be used. For a given depth d and calibratedmatrices P_(od), Q_(od), P_(cd) and Q_(cd), the estimated scene,{circumflex over (x)}_(d), may be obtained by solving the Tikhonovregularized least squares problem:

{circumflex over (X)} _(d)=arg min_(X) _(d) ∥P _(od) X _(d) Q _(od) ^(T)+P _(cd) X _(d) Q _(cd) ^(T) −Y∥ ₂ ²+λ₂ ∥X _(d)∥₂ ².  (4)

Further, for sparse scenes such as a double slit, {circumflex over(x)}_(d), may be obtained by solving the Lasso problem:

{circumflex over (X)} _(d)=arg min_(X) _(d) ∥P _(od) X _(d) Q _(od) ^(T)+P _(cd) X _(d) Q _(cd) ^(T) −Y∥ ₂ ²+λ₁ ∥X _(d)∥₁.  (5)

For the previously discussed 3D case, the Lasso problem is:

{circumflex over (X)} _(D)=arg min_(X)(Σ_(d=1) ^(D) ∥P _(od) X _(d) Q_(od) ^(T) +P _(cd) X _(d) Q _(cd) ^(T) −Y∥ ₂ ²)+λ₁ ∥X _(d)∥₁.  (6)

In equations 4-6, the second term is a regularization term in which λ₁and λ₂ control a tradeoff between fidelity and regularization.

Other methods may alternatively be used to invert equations 2 and 3,without departing from the invention. For example, if the calibrationmatrices are well-conditioned, the simple least-squares problem

{circumflex over (X)} _(d)=arg min_(X) _(d) ∥P _(od) X _(d) Q _(od) ^(T)+P _(cd) X _(d) Q _(cd) ^(T) −Y∥ _(F) ²,  (7)

where ∥ ∥_(F) denotes the Frobenius norm, may be solved.

Iterative techniques may be used to solve these above optimizationproblems.

For example, gradient methods such as Nesterov's gradient method or afast iterative shrinkage-thresholding algorithm (FISTA) or any othermethods suitable for solving the above optimization problems may beused.

In one or more embodiments of the invention, it may be assumed thatunforeseen debris or other imperfections may be incorporated into thesystem. The imperfections may cause aberrations or other artifacts.Embodiments of the invention may remove such aberrations by assuming thepresence of error in the received EM radiation from the scene:

Y={tilde over (Y)}+E  (8)

where {tilde over (Y)} is the aberration free measurement and E is theaberration. Assuming that the aberration is localized, E will be asparse matrix. Based on these assumptions, {tilde over (Y)} may beestimated as:

min_({tilde over (Y)}) ∥{tilde over (Y)}∥ _(*) +β∥{tilde over (E)}∥ ₁,subject to {tilde over (Y)}+E=Y.  (9)

An aberration free estimate of the scene radiance may be generated usingthe above equations by substituting {tilde over (Y)} for Y.

In one or more embodiments of the invention, the calibration matricesmay be determined by modeling of the modulation and diffraction of EMradiation due to the masks (120). For example, the masks (120) may bemodeled computationally to determine how light from a scene (100) isdistributed onto the Image sampler and thereby determine the calibrationmatrices. In one or more embodiments of the invention, the calibrationmatrices may be determined by the method shown in FIG. 8.

Numerous variations on the system shown in FIG. 1 are possible withoutdeparting from the invention. FIG. 2 shows another embodiment of alens-free imaging system in accordance with embodiments of theinvention. The system shown in FIG. 2 may be configured to imagetranslucent and/or transparent specimens. Like-numbered elements shownin FIG. 2 perform the same function as like-numbered components inFIG. 1. The system of FIG. 2 includes radiation source(s) (110) disposedon a side of the scene (100) opposite the masks (120). The radiationsource(s) (110) may be, for example, an incandescent source, a lightemitting diode, or any other EM radiation producing device. In one ormore embodiments of the invention, the illumination sources (110) may beambient illumination.

Radiation generated by the radiation source(s) (110) may be transmittedthrough the scene (100) to the image sampler (140). Transmission throughthe scene (100) may modulate the radiation and thereby encodeinformation of the scene into the radiation received by the imagesampler (140). The image sampler (140) shown in FIG. 2 may includesensing regions (142) and may not include EM radiation generationregions (141, FIG. 1).

An exemplary lens-less imaging system, in accordance with one or moreembodiments of the invention, may be manufactured as follows. A 100 nmthin film of chromium is deposited onto a 170 μm thick fused silicaglass wafer and is subsequently photolithographically patterned. Thechromium is then etched, leaving the MURA pattern with a minimum featuresize of 3 μm. The wafer is diced to slightly larger than the active areaof the imaging sensor. The imaging sensor may be, for example, a SonyIMX219 sensor, which provides direct access to the surface of the baresensor. The diced amplitude mask is rotationally aligned to the pixelsof the imaging sensor under a microscope to ensure that the previouslydiscussed separability assumption is appropriate. The amplitude mask isthen epoxied to the sensor using a flip chip die bonder. To filter lightof a particular wavelength, an absorptive filter may be added by cuttingthe filter to the size of the mask and by attaching it, using, forexample, an epoxy and a flipchip die bonder in the same manner. Thedevice may further be conformally coated with a <1 μm layer of parylenefor insulation.

Further, because the separability of the mathematical model is based onlight propagation through a homogeneous medium an attempt is made toreduce large changes of the refractive index between the scene and theinterface of the lens-less imaging system. A large change of therefractive index (e.g., due to an air-to-glass transition) may result ina mapping of lines in the scene to curves at the sensor plane, thusreducing the applicability of the separability assumption. In contrast,relatively smaller changes of the refractive index (e.g., inwater-to-glass and/or biological sample-to-glass interfaces) onlyminimally affect the model. For calibration and image capturing, arefractive index matching immersion oil (e.g., Cargille 50350) may,thus, be applied between the surface of the mask and the target.

FIG. 8 shows a flowchart of a method in accordance with one or moreembodiments of the invention. The method depicted in FIG. 8 may be usedto establish a transfer function of a lens-free imaging device inaccordance with one or more embodiments of the invention. One or moresteps shown in FIG. 8 may be omitted, repeated, and/or performed in adifferent order among different embodiments.

Prior to reconstructing a 3D volume from a single lens-less imagingsystem measurement, a calibration may need to be performed in order toidentify the transfer functions (or calibration matrices) {P_(od),Q_(od), P_(cd), Q_(cd)}^({d=1, 2, . . . , D}). As previously noted,these transfer functions, in accordance with an embodiment of theinvention, are separable. Accordingly, to estimate these transferfunctions for a particular single lens-less imaging system, images froma set of separable calibration patterns may be captured, as illustratedin FIGS. 7A-7C. Because the calibration patterns that are displayed areseparable, each calibration image depends solely on either the rowoperation matrices {P_(od), P_(cd)}^({d=1, 2, . . . , D}) or the columnoperation matrices {Q_(od), Q_(cd)}^({d=1, 2, . . . , D}) thussignificantly reducing the number of images to be processed for acalibration. Using a truncated singular value decomposition (SVD), therows and columns of {P_(od), QP_(cd), P_(cd),Q_(cd)}^({d=1, 2, . . . , D}) may be estimated as follows.

The calibration of the lens-less imaging system, in accordance with oneor more embodiments of the invention, relies on the characteristic thatif the scene is separable (rank-1), then the measurement obtained by thelens-less imaging system is rank-2. For example, if the scene has onlythe i^(th) row active (or illuminated), then the scene may be written asX_(i)=e_(i)1^(T), where e_(i) is a sequence of zeros with only thei^(th) element to be 1 and 1^(T) is a sequence of all 1 s. Then themeasurement of the lens-less imaging system may be written as:

Y _(i)=(P _(o) e _(i))(Q _(o)1)^(T)+(P _(c) e _(i))(Q _(c)1)^(T) =p_(oi) q _(o) ^(T) +p _(ci) q _(c) ^(T)  (9)

Here, p_(oi) and p_(ci) are the it columns of P_(o) and P_(c),respectively, and q_(o) and q_(c) are the sums of columns of Q_(o) andQ_(c), respectively. p_(Oi) and p_(ci) are orthogonal and may becomputed (up to a scaling factor) via the Singular Value Decomposition(SVD) of Y_(i) truncated to the two largest singular values. Since thesensor measurements are always positive, the truncated SVD of Y₁ yieldsone vector with all positive entries and another vector with bothpositive and negative entries. The positive vector may be assigned top_(oi), and the other vector may be assigned to p_(ci). By scanning therows of the scene, one may compute all the entries in P_(o) and P_(c).Similarly, the columns of Q₀ and Q_(c) may be calibrated by scanningalong the columns of the scene. Scanning the rows and columns of thescene may be physically done by translating a line slit, as shown inFIGS. 7A-7C. The transfer functions {P_(od), Q_(od), P_(cd), Q_(cd)} aredependent on the distance of the scene d and may be calibrated for eachdepth by first translating the line slit to the required depth and thenscanning the field of view. The number of calibration images needed is,thus, equal to sum of number of columns and number of rows of scene X.If the scene is of size N×N, then the number of calibration imagesneeded is 2N. As those skilled in the art will appreciate, this issignificantly less than the N² number of calibration images needed for ageneralized linear model.

The calibration procedure may be independently performed for each depthplane d. Given a measurement Y and the separable calibration matrices{P_(od), Q_(od), P_(cd), Q_(cd)}^({d=1, 2, . . . , D}), a regularizedleast squares reconstruction algorithm may be used to recover either a2D depth plane X_(d) or an entire 3D volume X_(D). The gradient stepsfor this optimization problem are computationally tractable due to theseparability of the model.

The following flowchart illustrates the steps performed when executingthe above operations. The steps are illustrated with reference to FIGS.7A-C, which illustrate lens-less imaging system (700) being calibratedusing a line slit (704), illuminated by a light source (702). The lineslit, illuminated by the light source, generates a light line. In anexemplary embodiment, a 5 μm wide line slit, fabricated in a 100 nm filmof chromium on a glass wafer is used. An LED array (green 5050 SMD)located ˜10 cm away from the line slit is further used as a lightsource. To ensure that the light passing through the calibration slit isrepresentative of a group of mutually incoherent point sources, awide-angle diffuser (Luminit 80°) is placed between the line slit on thewaver and the light source. In FIGS. 7A-7C, the lens-less imaging systemremains static, while the calibration slit, diffuser and LED array aretranslated with linear stages/stepper motors, separately along thex-axis and y-axis. For the calibration, the horizontal and verticalslits are translated over the field of view of the lens-less imagingsystem, determined by the acceptance profile of the pixels in theimaging sensor. In the exemplary system, a translation step distance of2.5 μm was repeated at different depth planes ranging from 160 μm to1025 μm (to perform a calibration for 3D imaging), while a translationstep distance of 1 μm was used for a single depth of 150 μm (to performa calibration for 2D imaging).

Turning to FIG. 8, in Step 800, the light line is aligned to thelens-free imaging system. The light line may be, for example, a slit ina metal film and a light source disposed on a side of the metal filmopposite the lens-free imaging system, as shown in FIGS. 7A-7C. Othertypes of light lines may be used without departing from the invention.

In Step 810, the light line is translated along the lens free imagingsystem, as illustrated in FIG. 7B, and electrical signals are recordedduring the translation. As the light line is translated, snap shots ofthe electrical signals are recorded as the light line is aligned witheach row of sensing regions of the image sampler.

In Step 820, the light line is rotated to be aligned with a column ofsensing regions of the image sampler that is perpendicular to a row ofsensing regions of the image sampler that the light line was alignedwith before the rotation. The light line is translated along the lensfree imaging system and the electrical signals are recorded during thetranslation. As the light line is translated, snap shots of theelectrical signals are recorded as the light line is aligned with eachcolumn of sensing regions of the image sampler.

Thus, during steps 810 and 820, a total of 2N snap shots of theelectrical signals, generated by the image sampler, are recorded, whereN is the number of sensing regions along an axis of the image sampler,assuming that each axis of the image sampler has the same number ofsensing regions along each axis.

In Step 830, the calibration matrices P_(od), P_(cd), Q_(od), and Q_(cd)are determined using the recorded signals, as previously described.

In Step 840, the calibration matrices are stored. The calibrationmatrices may be used to directly calculate the scene estimate usingelectrical signals generated by the image sampler.

Execution of Steps 800-840 establishes the calibration matrices for asingle depth plane, suitable for imaging samples in the depth plane. If3D volumes are to be imaged, Steps 800-840 may be repeated for otherdepth planes. Accordingly, in Step 850, the depth plane may be changed,e.g., by modifying the distance between the line slit (704) and thelens-less imaging system (700). Steps 800-850 may be repeated for anynumber of depth planes, and the obtained calibration matrices {P_(od),Q_(od), P_(cd), Q_(cd)}^({d=1, 2, . . . , D}) may subsequently be usedfor the imaging described in FIG. 9. The calibration matrices may remainvalid, as long as the geometry of the lens-free imaging device remainsunchanged.

FIG. 9 shows a flowchart of a method in accordance with one or moreembodiments of the invention. The method depicted in FIG. 9 may be usedto generate an image of a scene using a lens-free imaging device inaccordance with one or more embodiments of the invention. One or moresteps shown in FIG. 9 may be omitted, repeated, and/or performed in adifferent order among different embodiments.

In Step 900, electrical signals from an image sampler are obtained by animage sampler processor while the image sampler is exposed to EMradiation from a scene that is disposed near the masks of the imagesampler. For example, the image sampler may be exposed to EM radiationfor 10 milliseconds and the scene may be a finger pressed against themasks of the image sampler.

In Step 910, the image sampler processor generates an estimate of thescene based on the obtained electrical signals and a transfer function.The transfer function may be stored in the image sampler processor. Thetransfer function may include the calibration matrices determined usingthe method of FIG. 9. The transfer function may be determined prior toperforming the method of FIG. 9 and may be, for example, one of thepreviously discussed equations 4-7, which enable reconstruction of 2Dplanar images or 3D volume images.

While the above description of generating an image of a scene has beenfor single images or stacks of images that form a 3D volume, embodimentsof the invention may include generating a number of images using theabove method at different points in time to generate a video. In one ormore embodiments of the invention, an image of a scene at a first pointin time may be used to improve the speed of reconstruction of a secondimage scene at a second point in time. For example, there may be littledifference between images of a scene at different points in time thatare temporally separated by small amounts, e.g., fractions of a second.An image of a scene at a first time that is temporally separated from animage as a scene at a second time may substantially reduce thecomputation power and time required to estimate a scene when compared togenerating an image of a scene without a previously generated image ofthe scene.

One or more embodiments of the invention may provide one or more of thefollowing advantages: (i) a system in accordance with embodiments of theinvention may be used to obtain high-resolution 2D and 3D images, e.g.,at micron-resolution, from a single captured frame; (ii) a system inaccordance with embodiments of the invention may be significantly morecompact than lens-based systems; (iii) a system in accordance withembodiments of the invention may have a significantly wider field ofview than comparable lens-based systems; (iv) a system in accordancewith embodiments of the invention may have a very high light collectionability; (v) a system in accordance with embodiments of the inventionallows the rapid computation of an image; and/or (vi) a system inaccordance with embodiments of the invention may be less expensiveand/or simpler to produce when compared to lens-based imaging systems.Systems in accordance with one or more embodiments of the invention aretherefore particularly suited for a variety of applications includingin-vitro and in-vivo microscopy. A system may be used, for example, for3D fluorescence imaging of large volumes spanning, e.g., multiple cubicmillimeters of tissue, and/or as implantable imaging devices thatminimize tissue damage while providing a wide field of view, usingpotentially multiple adjacently located lens-less imaging devices. Othermicroscopy applications include various bright-field, dark-field,reflected-light, phase contrast, bioluminescence and calcium signalingmicroscopy techniques. Systems in accordance with one or moreembodiments of the invention may further be ideally suited forbiometrics applications. Using larger imaging sensors or arrays ofsmaller imaging sensors, the lens-less imaging devices may be employedto capture biometrics including, but not limited to, fingerprints, veinpatters, faces, iris patterns, etc.

While the invention has been described with respect to a limited numberof embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that other embodiments can be devised whichdo not depart from the scope of the invention as disclosed herein.Accordingly, the scope of the invention should be limited only by theattached claims.

What is claimed is:
 1. A lens-free imaging system, comprising: an imagesampler; a radiation source; a mask disposed between the image samplerand a scene; and an image sampler processor that: obtains signals fromthe image sampler that is exposed, through the mask, to radiationscattered by the scene, wherein the scene is illuminated by theradiation source; and estimates an image of the scene based on thesignals from the image sampler, processed using a transfer function thatrelates the signals and the scene.
 2. The lens-free imaging system ofclaim 1, wherein the mask comprises a two-dimensional pattern of regionsthat alter the radiation received, through the mask, by the imagesampler and from the scene.
 3. The lens-free imaging system of claim 2,wherein the regions comprise at least one selected from a group ofradiation path length changing regions, radiation amplitude changingregions, radiation phase changing regions, and regions with radiationscattering characteristics.
 4. The lens-free imaging system of claim 2,wherein the two-dimensional pattern is composed of a cross-product oftwo one-dimensional functions, thereby making the two-dimensionalpattern separable.
 5. The lens-free imaging system of claim 4, whereinthe transfer function comprises a first term for a hypothetical “open”mask and a second term for a hypothetical “coding” mask, based on theseparable two-dimensional pattern.
 6. The lens-free imaging system ofclaim 1, wherein the estimate of the image of the scene is performed fora single depth, thereby producing a planar two-dimensional imageestimate.
 7. The lens-free imaging system of claim 1, wherein theestimate of the image of the scene is performed for multiple depths,thereby producing a three-dimensional image estimate.
 8. The lens-freeimaging system of claim 1, wherein the scene is one selected from agroup consisting of an in-vitro and an in-vivo sample.
 9. The lens-freeimaging system of claim 8, wherein one selected from a group consistingof fluorescence, bioluminescence, calcium signaling, bright-field,dark-field and phase-contrast microscopy is performed on the scene. 10.The lens-free imaging system of claim 8, wherein the scene is abiometric.
 11. The lens-free imaging system of claim 1, wherein thescene is disposed adjacent to the mask, at a distance less than 5 mmfrom the mask.
 12. The lens-free imaging system of claim 1, wherein theimage sampler is configured to capture at least one selected from agroup consisting of visible, infrared, ultraviolet, microwave, andionizing radiation.
 13. The lens-free imaging system of claim 1, whereinthe image sampler is configured to capture a sequence of images atdifferent points in time to generate a video.
 14. The lens-free imagingsystem of claim 1 further comprising a glass plate disposed between themask and the scene, wherein a sample that forms the scene is placed overthe glass plate.
 15. The lens-free imaging system of claim 14, whereinthe glass plate is one selected from a group consisting of a slidecomprising a biological sample and a flow channel for flow cytometry.16. The lens-free imaging system of claim 1 further comprising a secondimage sampler, wherein the second image sampler is disposed adjacent tothe first image sampler in order to capture a scene that extends acrossthe first and the second image sampler.
 17. A method for generating animage of a scene using a lens-free imaging system, the methodcomprising: obtaining signals from an image sampler that is exposed,through a mask, to radiation scattered by the scene; and estimating theimage of the scene, based on the signals from the image sampler,processed using a transfer function that relates the signals and thescene.
 18. The method of claim 17, wherein the mask comprises a patternthat enables the separation of the mask into a hypothetical open maskand a hypothetical coding mask, and wherein the transfer functioncomprises additively superimposed terms representing the hypotheticalopen mask and the hypothetical coding mask.
 19. The method of claim 17further comprising obtaining the transfer function, using a calibration,prior to estimating the image.
 20. The method of claim 19, whereinobtaining the transfer function comprises: obtaining a series ofcalibration signals from the image sampler that is exposed, through themask, to radiation scattered by a series of separable calibrationpatterns; and identifying the transfer function using the series ofcalibration signals and the series of separable calibration patterns,wherein the identified transfer function establishes a relationshipbetween the series of separable calibration patterns and the series ofcalibration signals, thereby enabling estimation of the scene, using thesignals.